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INTERNET-LINKING SCANNER 

Background and Summary of the Invention 
Various technologies have been proposed for linking from physical media (e.g., 
5 magazine advertisements) to the internet. 

The present assignee offers a service (MediaBridge™) that relies on digital 
watermarking technology to surreptitiously convey a plural bit identifier within the 
artwork of an advertisement. This identifier is read by software in a user's computer and 
forwarded to a remote database. The remote database identifies a URL corresponding to 
10 the identifier, and provides the URL back to the user's computer - permitting a browser 
on the user's computer to display the URL-identified web page. This system, and the 
associated digital watermarking technology, is further detailed in patent 5,862,260, in 
copending applications 09/343,104, 09/503,881, 60/158,015, 60/163,332, 60/164,619, 
and in application 09/531,076, filed March 20, 2000, entitled SYSTEM FOR LINKING 
15 FROM OBJECTS TO REMOTE RESOURCES. 

Another approach relies on publication of bar code identifiers in advertisements. 
A barcode reading pen, or similar device, is used to sense the barcode, which is then 
processed to provide a corresponding URL. Such technology is disclosed, e.g., in the 
present assignee's application 09/452,021, and in issued patents 5,978,773, 5,971,277, 
20 5,918,214, 5,933,829, and 5,905,251. 

In accordance with the present invention, a scanner is provided with means to 
discern an identifier (encoded as a digital watermark, a barcode, or both) from scan data. 
The scanner can provide this identifier to an auxiliary device (e.g., a local or remote 
computer), or can utilize the information internally (e.g., by presenting same to the user). 
25 In a particular implementation, a hand scanner - such as the HP CapShare 920 

scanner - is adapted (i.e., programmed) for this purpose. 

The foregoing and additional features and advantages of the present invention will 
be more readily apparent from the following detailed description, which proceeds with 
reference to the accompanying drawings. 
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Brief Description of the Drawings 
Fig. 1 is a block diagram of a prior art scanner that can be programmed to practice 
the techniques of the present invention. 

Fig. 2 shows an object being scanned along an arced path. 
5 Fig. 3 shows how object warping may be detected. 

Fig. 4 shows use of binocular processing to determine certain object attributes. 

Detailed Description 
The reader is presumed familiar with the workings of the HP CapShare scanner. 
10 Such information has been published in the technical literature, e.g., Allen, Ross R. et al, 
"Processes for Freehand Image Capture: HP CapShare Technology," The Society for 
Imaging Science & Technology PICS Conference, pp. 43-46, March 26, 2000. 

As shown in Fig. 1, the CapShare scanner 10 includes a CPU 12, a memory 14, a 
linear CCD array 16, first and second 2D CCD arrays 18a, 18b, a user interface 
15 (including an LCD screen and associated buttons) 20, and an interface 22 (e.g., a serial 
port and an IRDA port/transceiver) to an auxiliary device 24. Although not particularly 
shown, each of the CCD sensors has an associated LED system for illuminating the 
object being imaged. 

A further enhancement to the CapShare scanner is to provide a wireless internet 
20 interface for interface 22, permitting direct communication between the device 10 and the 
internet. 

The linear sensor 16 operates in a conventional manner, acquiring successive 
pixilated line-scans of imagery under the scanner at a pixel data rate of about 18 
Mbits/second (permitting full accuracy scanning even at instantaneous scanner motion 
25 speeds of up to 16 inches per second). 

The two 2D CCDs 18a, 18b, are spaced apart, adjacent the linear sensor, and are 
used to track the scanner's movement. The areas scanned by these CCDs are illuminated 
obliquely by IR light, highlighting microscopic media surface features. The CPU 
identifies patterns in the surface features detected by each of these CCDs, and tracks their 
30 movement from one frame to the next to discern the movement of the two CCDs. By 
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knowing the movement of the two CCDs, the movement of the scanner itself can be 
determined. This scanner motion information is used to re-map the swathed scan data 
acquired from the linear scanner array into composite pixel data with uniform scanline 
spacing. This remapped data is the final scan data that is typically provided to the end 
5 user or application. 

The sizing of the linear and 2D CCD arrays is left to the designer. If 600 dpi line 
scan resolution is desired across a four inch scan swath, a 2400 element linear CCD 
would be used. The 2D CCDs are typically smaller in extent, e.g., having an aperture of 
0.25 inches x 0.25 inches. The spacing of the CCD elements is selected based on the size 
10 of surface features to be tracked. 

The memory 14 includes both RAM and ROM. The RAM is used both for raw 
data storage, and for storage of final results. The scanner's operating system, pattern 
matching and data processing algorithms, and other program code are stored in the ROM. 

In accordance with the illustrated embodiment, the ROM code also includes 
15 instructions for examining the final-result scan data (i.e., after re-mapping) for watermark 
data. In one particular embodiment, this is a two-step process. The final-result data is 
first screened by the CPU to determine whether it has certain hallmarks associated with 
watermarked data (e.g., the presence of a calibration signal, as detailed in the cited 
applications). If such hallmarks are found, a more computationally-intense watermark 
20 decoding algorithm is performed (e.g., of the sort described in the cited applications). By 
checking for watermark information in such a two-step process, CPU time isn't spent 
needlessly trying to extract a watermark from image data that apparently has no 
watermark. 

In some embodiments, the watermark processing operation(s) occurs without user 
25 intervention - each time the raw scan data is processed and remapped into final-result 
form. In other embodiments, the watermark processing is invoked in response to a user 
command entered through the user interface 20. In still other embodiments, the 
watermark processing is invoked in response to a command provided to the scanner from 
an associated auxiliary device 24 (e.g., a local personal computer, a remote server 
30 computer, a specialized internet appliance, etc.). 
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In some embodiments, the decoded watermark payload data is transferred to an 
associated auxiliary device whenever such data is detected. In other embodiments, the 
data is not sent unless the auxiliary device 24 first asks for it. Such an auxiliary device 
request may be made by a software program that is executing on the auxiliary device, 
5 e.g., through an API call. 

A watermark decoded by the scanner can be presented to the user on the scanner's 
LCD display 20. Or the scanner can forward the decoded watermark to a remote device 
24, which can then reply with supplemental data for presentation to the user. (Such 
arrangements for providing internet-based content and controls in response to decoded 

10 watermarks are more particularly detailed in the present assignee's cited patent 

applications. In such applications, the scanner may serve the role of the client device, or 
the "originating device" as that term is used in the System For Linking From Objects To 
Remote Resources patent application.) 

In some embodiments, the information obtained by the 2D CCDs 18a and/or 18b 

15 can be used to augment the information provided to the CPU so as to aid in the 
watermark detection/decoding process. For example, the 2D sensors provide 2D 
information immediately - without the time delay associated with remapping the ID data 
from the linear CCD array into final form. This immediate 2D information may be 
analyzed for the presence of a calibration signal - perhaps shortening or obviating 

20 analysis for the calibration signal in the final data. 

The 2D data can also be used to characterize the texture of the imaged substrate. 
Knowing the substrate texture allows the CPU to apply filtering or other watermark- 
enhancing/image enhancing steps as best fits the particular circumstance. 

The 2D data also allows the CPU to assess the relative quality of different regions 

25 of line-scan data in the final scan data. In Fig. 2, an object 30 is scanned by a scanner 10 
traveling an arced path, yielding successive lines of raw scan data 28. The geometry of 
the arc is revealed by data collected by the two 2D CCDs (which serve as motion 
encoders). Where the successive lines are spaced closer together (as at region A), the raw 
scan data is of higher resolution (relatively "oversampled"). In contrast, where the 

30 successive lines are spaced further apart (as at region B), the raw scan data is of lower 
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resolution (relatively "undersampled"). In its screening or analysis for subtle watermark 
data, the CPU can be instructed to look first in region A, reasoning that the higher quality 
of the raw data in this region is most likely to include usable watermark information. (In 
some embodiments, watermark screening and/or detection may proceed by reference to 
5 the raw scan line data, rather than waiting for the data to be remapped into final form.) 

The two CCDs 18a, 18b also permit various binocular processing to be performed 
so as to enhance, or allow for more intelligent processing, of the linear scan data. 
Consider, for example, the case where the image being scanned doesn't convey a 
perfectly planar (e.g., rectilinear) signal. The substrate could be warped, or the image 

10 may have been printed in a skewed manner, or some other transformation may have taken 
place that creates an image that changes orientation/scale over its extent. One such 
example is shown in Fig. 3, which depicts a watermark calibration signal (here presented 
as an overt grid, for clarity of illustration). Each of the two 2D CCDs 18a, 18b senses a 
differently-oriented calibration signal. By reference to differences in the data sensed by 

15 the two spaced-apart CCDs, the CPU can infer scale, rotation, or other image 

transformation at intermediate points, and apply a compensation to the scan data so as to 
counteract such distortion. 

Such compensation need not be based on a watermark calibration signal. Any 
other optically-sensed attribute at the two spaced-apart points may be used to infer the 

20 same, or related attributes, at intermediate positions, permitting appropriate 
compensation(s) to be applied. 

Another application of the spaced-apart sensors 18 is to bi-optically determine the 
distance from the scanner to the object being imaged. Consider Fig. 4, which shows a 
scanner 10 with two spaced-apart CCD sensors 18a, 18b, imaging a warped object 32 

25 (e.g., a warped paper substrate, or a curved object such as a drink can). In the depicted 
example, the first CCD 18a detects a very crisp image of surface texture or imagery, 
whereas the second CCD 18b detects an image similar to the first CCD, but blurred - as 
if low-pass filtered. Software instructions for CPU 12 can cause the scanner to recognize 
that the part of the object 32 under CCD 18a is immediately adjacent the scanner (and 
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adjacent the nearby portion of the linear CCD array 16), whereas the part of the object 
under CCD 18b is spaced somewhat from the scanner. 

Having information about the spacing of the object 32 from different parts of the 
linear sensor array permits certain compensations to be made in the data collected by the 
5 linear scanner. In the case of watermark detection, the CPU may recognize that data 
from the end of the linear scanner 16 remote from the object 32 will be devoid of 
meaningful high frequency information. In many watermark detection algorithms, the 
CPU can disregard data from such portion of the linear scanner - allowing it to focus on 
portions of the image data with the necessary high frequency components, thus producing 

10 more reliable results more quickly. 

In other watermarking systems, two watermarks may be present - one conveyed 
in reliance on high frequency image components, and another encoded mostly with low 
frequency image components (i.e., in anticipation of circumstances where the object is 
imaged slightly out-of-focus). Upon recognizing that the linear scan data is weak in high 

15 frequency components, the CPU can be programmed to look just for the low frequency 
watermark data. 

Just as the scanner 10 can perform watermark decoding, it can similarly perform 
barcode decoding. Conventional pattern-recognition algorithms can be applied by CPU 
12 to either the raw or final scan data to identify barcode patterns. Once the pattern is 
20 identified, decoding is straightforward by applying known barcode alphabets. As in the 
watermark case, barcode decoding can be performed autonomously, or in response to 
user/auxiliary device command. Similarly, the decoded barcode data can be provided to 
the auxiliary device whenever detected, or in response to an auxiliary device query. The 
2D data can likewise be used to augment the information provided to the CPU so as to 
25 aid in the barcode detection/decoding process. 

In similar fashion, the above-described benefits associated by use of the two 2D 
CCDs can similarly be applied in the barcode context. 

As earlier described, once the identifier information is extracted from the image 
data, internet links can be based thereon to provide supplemental information, e- 
30 commerce opportunities, etc. In many implementations, the scanner UI 20 is used to 
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present this supplemental information to the user, e.g. by software instructions that render 
HTML instructions for presentation on the UI display screen. The UI controls (e.g., 
buttons) can likewise be used to receive user instructions and commands, for linking back 
to the internet. 

5 To provide a comprehensive disclosure without unduly lengthening the 

specification, applicants incorporate by reference the patents and patent applications 
referenced above. 

Having described and illustrated the principles of the technology with reference to 
specific implementations, it will be recognized that the technology can be implemented in 

10 many other, different, forms. For example, while reference has been made to software 
and program code, in other embodiments such functionality can naturally be provided by 
dedicated hardware devices. Similarly, while the technology has been described in the 
context of a dedicated hand scanner device, in other embodiments the technology can be 
implemented in other forms. Naturally, substitution of components and technologies is 

15 expected (e.g., CMOS sensors for CCDs). 

The particular combinations of elements and features in the above-detailed 
embodiments are thus exemplary only. Moreover, the interchanging and substitution of 
these teachings with other teachings in this and the incorporated-by-reference 
patent s/applications are also contemplated. 
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WE CLAIM 

1. In a scanner including a CPU, a memory, a linear sensor array, and first and 
second spaced-apart 2D sensor arrays serving as motion encoders, the CPU serving to 
process raw scan data collected by the linear sensor array from an imaged object into 

5 final scan data in accordance with scanner motion data provided from said 2D sensors, an 
improvement comprising software instructions in the memory causing the scanner to 
discern a machine-readable identifier from scan data acquired from the object. 

2. The scanner of claim 1, further comprising a wireless interface, and a display, 
10 said software instructions causing the scanner to relay the identifier by the wireless 

interface to a remote server, and for processing information returned through the wireless 
interface for presentation on said display. 

3. The scanner of claim 1 in which the software instructions cause the CPU to 
15 process data from the 2D sensor arrays for a purpose in addition to sensing scanner 

motion. 

4. The scanner of claim 3 in which said purpose includes beginning a watermark 
detection process before data from the linear sensor array is finally processed. 

20 

5. The scanner of claim 4 in which said purpose includes beginning to sense a 
watermark calibration signal. 

6. The scanner of claim 3 in which said purpose includes identifying portions of 
25 the data collected by the linear sensor array that are relatively more likely to include 

detectable identifier data. 

7. The scanner of claim 3 in which said purpose is to quantify an object surface 
characteristic, wherein a filter can be applied to the scan data in accordance therewith. 

30 



WYCidks 4/5/00 60154 



-9- 



EXPRESS MAILEL525676294US 



8. The scanner of claim 3 in which said purpose is to assess relative distance to 
the object from different portions of the scanner. 

9. The scanner of claim 3 in which said purpose is to quantify an affine distortion 
5 in the scan data, so that compensation may be applied therefore. 

10. The scanner of claim 1 in which the identifier is steganographically encoded 
as a digital watermark. 

10 11. The scanner of claim 1 in which the identifier is encoded as a barcode. 

12. A scanner comprising a linear sensor array, a CPU, an interface for coupling 
to the internet, and a user interface including a display screen and user controls, the 
scanner being adapted to transmit machine-readable data sensed from scanned objects to 
15 the internet, and for presenting to a user HTML information received back from the 
internet. 
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INTERNET-LINKING SCANNER 

Abstract of the Disclosure 
A scanner is programmed to extract a machine-readable identifier (e.g., encoded 
5 in watermark or barcode form) from an object, and transmit same to a remote computer 
(e.g., over the internet). The remote computer responds with supplemental information or 
e-commerce opportunities relating to the scanned object. This information may be 
presented on an LCD display built-in to the scanner. In some embodiments, auxiliary 
CCD sensors are provided in addition to the scanner's usual image sensor. These 
10 auxiliary sensors can be employed for various purposes, including identifying surface 
texture characteristics so that corresponding filtering/processing of scan data can be 
performed; detecting affine transformations of the object being imaged so appropriate 
compensations can be applied, etc., etc. 
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